Why Sparse? Fuzzy Techniques Explain Empirical Efficiency of Sparsity-Based Data- and Image-Processing Algorithms

In many practical applications, it turned out to be efficient to assume that the signal or an image is sparse, i.e., that when we decompose it into appropriate basic functions (e.g., sinusoids or wavelets), most of the coefficients in this decomposition will be zeros. At present, the empirical efficiency of sparsity-based techniques remains somewhat a mystery. In this paper, we show that fuzzy-related techniques can explain this empirical efficiency. A similar explanation can be obtained by using probabilistic techniques; this fact increases our confidence that our explanation is correct. I. FORMULATION OF THE PROBLEM Sparsity-based techniques are useful. In many practical applications, it turned out to be efficient to assume that the signal or an image is sparse; see, e.g., [1], [2], [3], [4], [5], [6], [7], [8], [9], [12], [13], [14], [17], [18]. In precise terms, sparsity means that when we decompose the original signal x(t) (or original image) into appropriate basic functions e1(t), e2(t), . . . (e.g., sinusoids or wavelets), i.e., represent this signal (or image) as a linear combination